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On  Calculating  Analytic  Centers 

A.  A.  Goldstein* 


This  note  was  motivated  by  papers  of  Renegar  and  Shub(8S)  and  by  Yc(S9).  We  apply 
Smale's(S6)  estimates  at  one  point  for  Newton’s  method  to  the  problem  of  finding  the 
analytic  center  of  a  polytope.  The  method  converges  globally  in  the  appropriate  norm. 
The  ideas  are  then  applied  to  obtain  a  possible  benchmark  for  path  following  methods. 

When  Smale’s  method  is  tractable  its  power  stems  not  only  from  the  fact  that  the  in¬ 
formation  is  concentrated  at  one  point.  There  are  2  norms  to  estimate,  not  3  as  in  the 
Kantorovich  estimate.  Moreover  no  estimate  of  the  inveme  of  the  derivative  operator  by 
itself  is  needed.  The  need  for  the  norm  of  the  inverse  by  itself  often  makes  for  coarse 

'jjt-MV  :,tos  .  . 

1.  Setting 

Let  A  denote'  an  m  by  n  orthonormal  matrix  of  rank  n  and  b  an  m  by  1  matrix. 
We  assume  that  m  >  it.  Denote  by  e  a  in  by  1  matrix  whose  components  arc  all  ones. 
Transposes  of  matrices  will  be  denoted  by  an  asterisk,  rows  of  a  matrix  by  superscripts, 
and  columns  by  subscripts.  The  euclidean  space  of  real  m-tuples  will  be  denoted  by  Em. 
If  u  6  Em  we  mean  by  diag(u)  the  diagonal  matrix  with  entries  ult.  The  dot  product 
corresponding  to  the  usual  norm  will  be  denoted  by  [  ,  ].  The  usual  norm  will  be  written 
as  Jj  || j.  E,,  will  be  also  be  renormed  under  a  dot  product  that  will  be  denoted  by  <  .  >. 
The  norm  arising  from  this  dot  product  will  be  written  as  ||  ||.  Let  P  be  a  polytope  with 
non-empty  interior  given  by  the  inequalities 

b  -  .4j-  >  0. 

Given  .r(i  in  the  interior  of  P  and  c  >  0.  we  seek  the  analytic  center  £  of  P  to  within  a 
tolerance  of  (.  Let  /?,(./' )  —  b,  —  .-ID  . 

Claim  1.  Let  X  be  the  smallest  ^er  exceeding 

log.,  (4.95  m  T  rnaxjRi(x0 ))  4-  log  > 

Then  if  X  steps  of  the  Xewton  sequence  are  generated  using  the  gradient  of  the  potential 
function  below 

+  supported  by  grants  XIH  RR01243-05  AXD  XPS  LMC-M4E1 


1  +  log 
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Ik.v-flb  <  e 

The  proof  of  the  claim  depends  on  the  following  ingredients. 

2.  Some  ingredients. 

The  potential  of  P  is  defined  by  the  expression 

m 

7 r{P)  =  max  J"J  H,(x)  :  x  E  P } 

1  =  1 


Aocession  For 
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t  *,  on 
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The  maximum  is  achieved  at  an  unique  point  called  the  analytic :  center  of  P.  (Ye  S7  j. 
We  shall  find  this  point  by  seeking  a  zero  for  the  gradient  of  the  logarithmic  potential 

m 

o(  J )  =  log  (R,(x) 

1=1 


Let  D  { x  )  =  diag{\ / Jt,(x)).  thus  D,,(x)  =  1  /R,(x). 

We  apply  Newton's  method  to  the  gradient  of  g>  which  we  denote  by  F.  F(xJ  may  be 
represented  by  the  matrix  .4*£>(.r)e.  and  F(  x )  E  E„.  The  kth  Frechet  differential  of  F  at 
x  can  be  identified  with  a  multi-linear  mapping  from  (E_\)k  to  En.  A  representation  of 
these  differentials  as  matrices  follows. 

F'(x)h ]  =  —.4*  D2{x)  diag{Ah\)t  =  —  A*  D{x)D(x)Ah\ 

F"  (x.  hi .  h.2)  —  —21A*  D(x)D2(x)  diagi  Ah>)  Ah\ 


F"'(x.  hi.h-2.hi)  =  —  31A*  D{x)D3(  x)  diag(Ah3  )  diag(  Ah2)  Ah: 

and 


F' !k>(  x .  h  | .  h-2 . In)  =  —k\A*  D{x)Dk{x)  diag(Ahk) diag(Ah-2)  Ah\ 


H  err 


=  -k\A'D(x)Q(x,hi . hk) 


\Qi 


.w/p{!!^(  x.  hi . hk  )||a 


«2 


= . =  l!h 


*  2 


1} 
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Theorem  1.  (Smale  80  )  Assume  F  is  an  analytic  map  between  real  Banach  spaces 

V,  that  is  the  Frechet  derivatives  F‘'l'\x)  exist  for  all  x  €  X  and  k=1.2.3 . G;ve 

X,  assume  that  the  inverse  of  F'{.r)  which  we  denote  by  F'_  ,(x)  exists.  Set 


H(x0)  =  ||irii(-ro)ir(-ro)||  and 


"/( -ro  )  =  *up 


\j]FL,(x0)F^(x0)\\^  :k>2 


If 


/3(x0)-\(x o)  <  .130707 


then  x o  is  an  approximate  root  of  F.  That  is,  the  Newton  sequence 


xk+i  =  J’k  -  F'_i(jrk)F(xk) 


is  well  defined  and  {xk}  converges  to  say  if.  a  root  of  F  at  the  rate: 


•r/t+i  ~Jrk\\  <  2{l-y  .i(x,) 


Moreover 


\‘rk  —  s  I.  <  7(0^  ^(3'0) 


(A) 


3.  Proof  of  Claim  1. 

Assume  x0  is  given  in  P(M).  The  matrix 

P(x0)  =  D(.r0)A{A*  D(x0)D(xu)A)-'  A*  D{x0) 

maps  each  point  in  E„,  to  its  closest  point  in  the  range  of  the  matrix  D(xq)A  . 
i|P(.r0)j|2  =  1.  We  renorm  En  by 

Ikll  =  l|PcAr||, 

Here  Dr  —  CD{x0)  with  C  =  1/S^um.  With  this  definition  we  get: 

A(x  u)  =  C||P(.r0)<'||2  =  "C/S5 

Also  ^  t 

<  C  ■•'up  ( 1 1 P*  t  *r  u  )||j  “ 1  )*up(\\Qk  Ah\  ||F  '  )  <  C 

1 

Thus  d(  )y(  .ro  )  <  -  <  1 3070  < 

O 
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X  and 
n  Xu  £ 


Hence 


Hence  by  Smale's  theorem  the  sequence  generated  by  the  Newton  algorithm  converges  to 
the  analytical  center  £  with  a  rate  given  by  (A)  in  Theorem  3.1  above. 


Since  <  x,  x  >=  [£>c-.4x.  DcAx]  >  C2  ||x  ||.]/max/?,( :c0)2 .  then 

lkl|2  <  C7?,(x0)||x|j 

Now  choose  N  so  that 

CRt{x0)\\xN -Z\\  <  € 


4.  Application  to  programming 


By  a  theorem  of  Ye  (89).  if  one  of  the  hyperplanes  of  P  is  translated  to  pass  thru  £  then 
the  resulting  polytope  P+  satisfies 


-(P+) 

~(P) 


< 


1 

t 


Consider  the  following  algorithm  for  linear  inequalities.  We  wish  to  solve  the  system 
h  —  Ax  >  0  if  this  is  possible.  Given  an  arbitrary  x0  choose  M  so  that  b  +  M  —  .4;r  >  0. 
Find  the  center  of  this  polytope  P(M).  Take  the  smallest  component  of  say  Rq(£). 

Begin  anew  with  the  polytope  P(M  -  i??(£)).  This  algorithm  has  a  worst  case  iteration 
count  of  0(m)  times  our  cost  of  getting  to  the  center. 

For  linear  programming  let  the  polytope  P  be  given  by  b  -  Ax  >  0  and  P(M)  the  polytope 
define  by  the  inequalities  for  P  together  with  the  inequality  M  -  [c  .  x]  >  0.  We  seek  the 
smallest  M  for  which  P(M)  is  non-empty.  We  first  find  the  center  f  of  the  polytope  P. 
We  then  find  the  intersection  of  the  ray  {.r  =  £  —  tc  :  i  >  0}  with  P  Translate  the  cost 
hyperplane  to  pass  thru  this  point.  Then  find  the  center  of  the  new  polytope  P(M). 


5.  Benchmark 

We  now  consider  the  possibility  of  starting  from  a  point  in  a  polytope  P(M)  and  moving 
to  the  center  of  a  neighboring  polytope  P(M  -  1/2 \/m)  by  Newton  steps. 

Assume  that  at  (x0.  4/0 ).  P,(x,M)  =  b,  +  Af0  —  .4’x  >  0.  We  seek  a  point  (x!..!/,  )  such 
that 


and  such  that 


Rt(bl  +  M\  —  A' X\)  >  0  (2) 


Let  Mi  =  M[j  —  l/2>/m.  Assume  that  the  value  of  X]  is  well  defined  and  given.  Otherwise 
P(M] )  is  empty  and  A/o  is  within  1/2 y/m  of  M*  the  optimal  value  of  M.  We  show  that 
(xo.A/o)  is  an  ’approximate  root’’  for  system  (I). 

Remark  The  matrix  (A  e)  has  rank  n+1. 

Proof  Because  of  our  boundedness  assumption  on  the  polvtopes.  the  system  of  inequalities 
Ax  >  0  is  inconsistent.  If  u  is  in  the  null  space  of  (A  e)  then  An  =  -un  +  if  ^  0.  a 

contradiction. 

In  matrix  notation  the  system  (1)  (after  scaling  the  second  entry  )  is 

/ 

F(x.M)  =  f  —A*D{x.  M )f. 


2-y/m 


e*(  £>(x.  A/)c  -  D(ti,  Mi  )c 


(/) 


Thus  we  see  that  -  F'(x.M)  may  be  generated  from  the  matrix 


D  —  ( .4] .  .4  2 . 4r,,.4n+i)  where  An+1  —  — 

Assume  that  ( _4i .  A> . 4„  )  is  rescaled  if  necessary  so  that  j|2?||  <  1.  By  the  Remark  we 

see  that  B  has  rank  n+1.  Thus  Claim  1  holds  for  this  case  as  well.  If  we  are  satisfied  with 
a  reduction  of  this  will  happen  in  N  steps  by  the  claim  with  e  set  to  1/Gy/rn.  We 

have  then  the  following  result:  (not  an  algorithm  but  a  benchmark) 

Claim  2.  We  tire  given  a  point  (x* .  A/* ),.  Let  A/*,  +  1  =  A/*  —  1/2 yfm.  If  P(A/;-+1)  is 
not  empty,  take  Xf:+i  for  its  center.  Let  the  system  (I)  be  run  with  Newtons'  method. 
Otherwise,  stop.  In  N  steps  A/*  will  be  reduced  by  at  least  1  /3yfm.  This  value  updates 
Mk+]  <md  the  corresponding  iterate  for  x  updates  x*.+).  Assume  the  optimal  M  say  A/* 
known.  Then  the  global  Newton  process  can  be  terminated  in  no  more  than  Q  steps,  where 

Q  >  3\/m(A/ii  -  A / * ) ( 1  +  log2 

G 


log.,  (4.95  m  *  m«x/?,(  x0  )J  +  lo<j>  ( 6\fm  ^ 


At  termination  A/y  is  within  1/2 \/m  of  M*  and  x\  is  an  approximate  root  for  system  (  I  1 
with  M%  replacing  M \  and  £  replacing  jp.  respectively. 

A  similar  result  holds  for  linear  programming. 

Bibliography 

James  Renegar  and  Michael  Shub  Simplified  complexity  analysis  for  Newton  lp  methods 
Cornell  OR  Report  no.  SOT  June  SS 

S.  Simile  Newton's  method  estimstes  from  data  at  one  point.  The  Merging  of  Disciplines... 
Springer-  Yerlag  19Sti  1S5-19C. 

V.  Ye  Karmarker's  algorithm  and  the  ellipsoid  method.  Operation  Res.  Letters  4.  177-182 
i 19S7 ) 

Y.  Ye  A  combinatorial  Property  of  Analytic  Centers  of  Polytopes  Dept  of  Management 
Science  U  of  Iowa  May  19S9 


INITIAL  DISTRIBUTION  LIST 


DIRECTOR  (2) 

DEFENSE  TECH.  INFORMATION  CENTER 
CAMERON  STATION 
ALEXANDRIA,  VA  22314 

DIRECTOR  OF  RESEARCH  ADMINISTRATION 
CODE  012 

NAVAL  POSTGRADUATE  SCHOOL 
MONTEREY,  CA  93943 

OFFICE  OF  NAVAL  RESEARCH 
CODE  4  2  2 AT 

ARLINGTON,  VA  22217 

PROFESSOR  ALLEN  GOLDSTEIN  (12) 

CODE  53GO 

DEPARTMENT  OF  MATHEMATICS 
NAVAL  POSTGRADUATE  SCHOOL 
MONTEREY,  CA  93943 

LIBRARY  (2) 

NAVAL  POSTGRADUATE  SCHOOL 
MONTEREY,  CA  93943 

DEPARTMENT  OF  MATHEMATICS 
CODE  53 

NAVAL  POSTGRADUATE  SCHOOL 
MONTEREY,  CA  93943 

CENTER  FOR  NAVaL  ANALYSIS 
4401  FORD  AVENUE 
ALEXANDRIA,  VA  22302-0268 


1 


